The present invention generally relates to an active attentional sampling technology for accelerating background subtraction from input videos. More specifically, the present invention relates to an active attentional sampling technology for accelerating background subtraction by removing background region from the input video and then applying foreground probability map and sampling mask according to temporal property, spatial property and frequency property of the input video in favor of the pixel-wise background subtraction algorithm.
The background subtraction algorithm is a process which aims to segment moving foreground objects from input video with subtracting background region. As computer vision technology is getting developed, computation time reduction issue in background subtraction algorithm becomes important in a systematic view, because the background subtraction is generally considered as a low-level image processing task to be processed with little computation. Further, the recent trend of bigger video sizes makes the computation time reduction issue more desperate.
Recently, background subtraction technology of pixel-based probabilistic model methods gained lots of interests and have shown good detection results. There have been many improvements in detection performance for these methods under various situations. However, heavy computation load of these methods results in long computation time, which renders real-time application impractical. Therefore, several approaches have been studied in order to reduce computation time of background subtraction algorithm.
The first type of approach is based on optimizing algorithms. The Gaussian mixture model (GMM) scheme proposed by Stauffer and Grimson works well for various environments. However, the GMM scheme shows slow learning rates and heavy computational load for each frame. D.-S. Lee, “Effective Gaussian mixture learning for video background subtraction,” TPAMI, 2005 made the convergence faster by using a modified schedule that gradually switches between two stage learning schemes. Z. Zivkovic and F. van der Heijden, “Efficient adaptive density estimation per image pixel for the task of background subtraction,” Patten Recognition Letters, 2006 achieved a significant speed-up by formulating a Bayesian approach to select the required number of Gaussian modes for each pixel in the scene. P. Gorur and B. Amrutur, “Speeded up gaussian mixture model algorithm for background subtraction,” AVSS 2011 modified Zivkovic's method by windowed weight update that minimizes floating point computations.
The second type of approach is using parallel computation in which multi-core processors using OpenMP or GPU are applied in a parallel form for enhancing computation speed. V. Pham et al., “GPU implementation of extended Gaussian mixture model for background subtraction,” IEEE RIVF, 2010 performed real-time detection even in full HD video using GPU. The second type of approach has successfully achieved speed enhancement, but requires parallel-processing hardware resources.
The third type of approach is using selective sampling. J. Park et al., “Hierarchical data structure for real-time background subtraction,” IEEE ICIP, 2006 proposed a hierarchical quad-tree structure to decompose an input image, by which computational complexity reduction is achieved. However, their algorithm may miss small objects because they randomly sample from a relatively large region. H.-K Kim et al., “Fast object detection method for visual surveillance,” IEEE ITC-CSCC 2008 presented a sampling mask designing method which can be readily applied to many conventional object detection algorithms. D.-Y. Lee et al., “Fast background subtraction algorithm using two-level sampling and silhouette detection,” IEEE ICIP, 2009 also proposed a two-level pixel sampling method. Their algorithms provide accurate segmentation results without flickering artifacts. However, these grid patterns still cause redundant operations in their algorithms.